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ABSTRACT 


Decision analysis methodology was applied to the problem 
of some project management trade-off decisions required for 
the Patrol Frigate (PF) Land Based Test Sites (LBTS). In 
particular, the question of whether to install or simulate 
the Mark 392 mod 2 fire control system, the AN/SPS-U9 radar 
and LM 2500 gas turbine engine at the LBTS was studied. The 
necessary decision analysis methodology required to 
formulate and solve the problem was presented. The results 
of this thesis indicated that the fire control system should 
be partially simulated, the radar should not be installed 
and the gas turbine engine should be simulated at their 


respective Land Based Test Sites. 
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I. INTRODUCTION 


A sum O oe ee ши завист? 


There is general recognition in the armed services that 
the management of projects concerned with weapons’ systems 
development and production is an increasingly complex and 
important task. The development and procurement costs of 
major weapons systems have steadily increased over the past 
ten years [Ref. 1}. Additionally, a need to reduce 
Department of Defense (DOD) spending has been identified 
many times by the President and other government officials 
as well as concerned citizens. These problems are placing 
an increasing burden on project management. The existence 
of cost and schedule over-runs and performance, 
maintainability, and reliability problems with the final 
weapons system hardware indicates there is room for 


improvement. 


There are several areas in project management where 
formal analysis can be of potential benefit to the project 
manager, to the Navy and to the researcher. One such area 
concerns how trade-off decisions are made among various 
factors such as cost, performance, and schedule. A good 
manager must have the ability to make reasonable judgments 
in these areas. He must determine the realizable 
alternatives and then select that alternative which best 
satisfies the overall objectives of the project. These 
judgments must often be made 1n highly uncertain 


environments. 


An approach to aid the manager in trade-off decisions, 


IO раса 000 US ed, in the past, is the use of decision 


analysis. Decision analysis 1S a systematic procedure to 
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logically describe, specify and balance all factors which 
affect a decision. Among the factors considered in decision 
analysis is the probability that a specific outcone will 
occur from the various possible consequences. The preferred 
alternative also depends on the decision~maker's preferences 
for those consequences. The techniques used in decision 
analysis may enable a project manager to break a relatively 
complex judgmental situation down to a series of simple 


decisions. 
A. PURPOSES 


The purposes of this thesis were fourfold. The first 
purpose was to demonstrate that decision analysis techniques 
can be used aS a project management tool. The second 
purpose was to provide the author with experience in 
conducting decision analysis in the project management 
environment. This was particularly useful because the 
research contained cases of multidimensional conseguences 
under uncertainty. The third purpose was to provide 
examples of project management decision problems which could 
be incorporated into the United States Naval Postgraduate 
School's Adminstrative Science Curriculun. The forth 
purpose was to generate interest in the use of decision 


analysis in project management decision-making. 
В. Davos CASED TEST OSLER TRADE-OFFS 


A current example of project management trade-off 
decision making is the problem of what constitutes an 
optimum mix of actual and simulated subsystems of the Patrol 
Frigate (FF) escort vessel which would be installed and 
tested at a land based test site. The testing must be 


SOLO PE production approval by the Defense 
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Systen Acquisition Review Council  (DSARC). Consequently 
three subsystems of the Patrol Frigate, the AN/SPS-49 radar, 
the Mark 92 mod 2 Fire Control System (FCS), and the LM 2500 
gas turbine engine, were selected as examples of how 
decision analysis can be demonstrated in this thesis. Four 


alternatives were considered. They are 
(1) installation of the actual equipment, 


(2) installation of some actual and some simulated 


equipment, 
(3) installation of entirely simulated equipment, or 


(4) no installation necessary at the land based test 


site, 
С. LINITATION 


There are some limitations to this thesis. It was 
written in the decision environment of the project manager. 
External influences such as political, social and cultural 
factors were not included. Some generalizations were made 
in order to maintain this thesis unclassified. Cost and 
Schedule parameters represent an order of magnitude and are 
the results of experienced judgments of personnel from the 
PF Ships Acquisition Project Office (PMS-399) located at the 
Naval Sea Systems Command Headquarters. A set of utility 
functions was developed for one individual concerned with 
the problem to illustrate the method and use of decision 
analysis. Finally, the preferences for consequences and the 
assessments of judgmental probabilities were provided by 
personnel in the PF Project Office. No refinements were 


made on these assessments from other sources. 
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D. THESIS ORGANIZATION 


this thesis is organized in the following manner. 
Chapter II presents a background of the Patrol Frigate Land 
Based Test Sites (LBETS). Decision analysis methodology is 
explained in Chapter III. This explanation includes 
discussion of the systematic steps used in formal decision 
analysis and the evaluation of multidimensional consequences 
under uncertainty. Formulation of the decision analysis 
procedures used in deciding the alternative that best 
satisfies the objectives of LBTS testing for the three PF 
Subsystems under consideration is described in Chapter IV. 
Chapter V provides the solution procedures and the 
recommended courses of action. Chapter VI summarizes the 
thesis, presents recommendations and provides suggestions 


for further research. 
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ІІ. BACKGROUND 


AR ie ee O A A шш nei ае шоор 


On 13 July 1971, Deputy Secretary of Defense David 
Packard created a major change in the DOD management of 
major defense system acquisitions with the issuance of 
Department of Defense Directive 5000.1. The Directive, 
titled "Acquisition of Major Defense Systems," applies to 
programs designated as major by the Secretary or Deputy 
Secretary of Defense. This Directive recognizes that 
sucessful development, production апа deployment of major 
defense systems are dependent on people, priorities and 
Clearly defined responsibility. To this end, the rolicy 
invoked by tnis document designates the program manager as 
the key individual accountable for a major acquisition 
program. He should have a charter with sufficient authority 
to accomplish his program objectives and sufficient tenure 
to accomplish his task. The layers of management between 
the program manager and his service should be minimized 
нет. 21. 


А. PF PROJECT 


If a program results.in an expenditure in excess of 50 
million dollars for research and development, or is in 
excess of 200 Deere sol larssduring”-preouuctı10nor dso urgent 
moman cnal Security it will be classified as a major 
acquisition program. The Patrol Frigate Project has been 


classified in this category. 


14 








1. objectiv 


— | P p 


The objective of the PF Project is to acquire by the 
late 1970's a class of ships which will provide, at least 
cost, the maximum improvement in the Navy's surface 
combatant capability to defend non-carrier forces against 


airborne and sub-surface attacks. 


The PF is an austere escort ship designed to provide 
maximum mission capability within the constraints of average 
follow ship cost of 47.7 million dollars (FY 73 dollars), a 
full load displacement not exceeding 3500 tons and total 
personnel accomodations not exceeding 185. Follow ship cost 
is defined to mean the unit production cost of ships built 


after construction of the lead ship. 


Hum KI ew eee eS ee ee ee O «чинии» ee eS oe 


There are four life cycle phases for a normal weapon 
System acquisition program. The first phase is the 
conceptual phase. The objective of this phase is to define 
the operational need. The system concepts which warrant 
development are explored. The second phase is the 
validation phase in which alternative system concepts are 
validated as a basis for determining whether or nct to 
proceed into full-scale development. The third phase is 
full-scale development. The objective or this phase is to 
design the systen, construct a full-scale prototype for test 
and evaluation and provide the documentation needed to 
produce the system. The final phase is the production phase 
in which the system is produced for operational use. Before 
one phase is permitted to proceed into the next phase, 
approval must be obtained from the Defense Systen 


Acquisition Review Council. 
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Тһе DSARC review process occurs at three decision 
point milestones during the normal life cycle of the system 


development. These points in time are defined as 


(a) DSARC I, which occurs between the conceptual and 


validation phase, 


(b  DSARC ІІ, which occurs between the validation 


and the full-scale development phase and 


(с) DSARC III, which occurs between the full-scale 


development and production phase. 


Eefore a project can sucessfully pass the three 
DSARC decisions, it must be increasingly better defineä and 
have minimized the technical uncertainty and program risks. 
Test and evaluation, to the extent it can be perfcrmed in 
each phase, is an important means for reducing such risk and 


uncertainty. 


ЧЕЧЕР ee es es es ee ee ep ee a eee ED ee VER шыша 


Department of Defense Directive 5000.3 [Ref. 3] 
establishes the рот су Ler the conduct of test and 
evaluation by the DOD components in the acquisiticn of 
defense systems. The general policy is that test and 
evaluation shall be commenced as early as possible and 
conducted throughout the acquisition cycle to assist in 


progressively reducing risks and assessing military worth. 


The Directive defines Development Test and 
Evaluation (DT&E) as the test and evaluation conducted to 
demonstrate that the engineering design and development 
process is ccmplete and that the design risks have been 


minimized.  DT£E commences as early in the development phase 








ав possible after DSARC II to demonstrate that technical 
risks have been identified and that solutions are in hand. 
Prior to DSARC III, the DT6E accomplished should ensure that 
the engineering is essentially complete and that all 
significant design problems have been identified and have 


solutions. 


Operational Test and Evaluation (OT6E) is conducted 
to evaluate the developed systen's operational 
effectiveness. The Directive specifies that acquisition 
prograus will ре structured so that at least an initial 
phase of Operational Test and Evaluation (IOT&E) will be 
accomplished prior to DSARC III adequate to provide a valid 
estimate of the expected system operational effectiveness 


and suitability. 


Department of Defense Directive 5000.3 further adds: 


"The, long, design, engineering, and 
construction period of a mäjor zu. MD WU погша 1 
ргес иде conpletion ot the ead. ship an 
accomplishment of test. therefore prior to the 
decision to proceed with follow ships. Іп lieu 
thereof, sucessive phases of Developmen Test and 
Evaluation ee and Operational Test and 
Evaluation (OT&£2) will be accomplished as early as 


practicable at test installations and on the lead 
Sharp W504 a SEES гарта 8) чгеписе# гискса ашо thereby 
mininize the need for modification to follow ships." 


eng due A SE Se ED u ЧЫР ee ER ER p EEE ee EE ER PA A A A RE eee See cee 


The DSARC II review for the Patrol Frigate Program 
was held on 31 August 1972. The Deputy Secretary of Defense 
issued a Decision Memorandum for the Secretary of the Navy. 
In this Memorandum, one of the decisions reached by the 
Deputy Secretary of Defense was that approval of follow ship 
producticn should be contingent upon adequate test and 
evaluation individually on subsystems and collectively at 
land based test sites (Ref. 4]. In order to satisfy the 


requirements contained jin Refs. 2 - 4 described above, the 
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Project Manager has ‘determined that the propulsion and 
combat systems for the Patrol Frigate should be prototyped 
at land based test sites (LBTS). 


The Propulsion System LBTS is now under construction 
at the Naval Shipyard in Philadelphia, Pennsylvania. The 
LBTS will consist of two LM 2500 gas turbine engines, one 
reduction gear complete with high speed clutches and brakes, 
one controllable reversible pitch propeller hub with dummy 
blades, one Shaft, a control system and a control station. 


Figure 1 illustrates the Propulsion system arrangement. 


The Combat System LBTS, is located at MacArthur 
Field, Leng Island, New York. Actual equipment to be 
installed consist of one AN/SPS-49 air search radar, one 
AN/SPS-55 surface search radar, a Mark 92 mod 2 Fire Ccntrol 
System, two AN/UYK-7 computers used as Weapons Control and 
Support Processors and miscellaneous switchboards and 
communications equipment. Functional simulators will be 
provided for the missiles, launchers, gun mounts,sonar and 
electronics countermeasure equipment (Ref. 5]. Figure 2 
illustrates a block diagram arrangement of the Combat System 
LBTS. It is to be noted that the location of the Combat 
System LBTS is five miles from the coast line, surrounded by 
trees and dwellings. This will limit low flying aircraft 
operations and prohibit the actual firings of guns or 


missiles [Ref. 6]. 
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Note: 


PF PROPULSION SYSTEM LBTS EQUIPMENT CONFIGURATION 


LM 2500 LM 2500 
GT ENG T ENG 


REDUCTION GEAR 





CONTROL 


STATION 





SHAFT 


The water brake 
is installed 
to aksorb 
the energ 
being developed 
by the main 
propulsion units WATER 
when іп 
O eration. BRAKE 
it is not 
intended for 
shipboard 
installation. 
CRP PROPELLER HUB 


FIGURE 1 
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PF COMBAT SYSTEM LBTS EQUIPMENT CONFIGURATION 





MK 92/2 FCS 
TRACKING 


ИК 92/20 ЕСЕ 
ILLUMINATION 
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FIGURE 2 
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В. А 1815 PROBLEM 


ғ 


The PF Development Concept Paper (Ref. 7] outlines the 


following critical test and evaiuation questions and issues: 


(1) Are the individual combat subsystems to be 
incorporated in the ship technically acceptable and 


operationally suitable and effective? 


(2) Does the integrated propulsion system installed at 
the Propulsion System BIS demonstrate sufficient 
performance and reliability to indicate that operational 


requirements will be met? 


(3) Is the integration of the various Combat Systens 
through their respective interfaces adequate, in terms of 
data transfer characteristics such as handling capacity, 


rate and quality, to meet operational, requirements? 


(4) Do those operational characteristics of the 
integrated Combat and Propulsion Systems, including 
reliability and maintainability, which can be estimated 
based on ¡OT8E, show a reasonable probability that the ship 
class when afloat will be able to satisfactorily accomplish 


the missicn for which it was designed? 


One of the decision problems, then, for the PF Project 
Manager was what the composition of subsystems should be in 
the Combat and Propulsion Systems LBIS so that "adequate 
test and evaluation" [ref. 4} with satisfactory results can 
be supplied to DSARC III, scheduled for Нау 1975. The 
installation and testing of these PF subsystens are to be 
accomplished to ensure that the critical test and evaluation 


questions outlined in the Development Concept Paper (Ref. 7] 
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аге answered at minimum cost. Additionally, the primary 
objective of the LBTS after DSARC III production approval 
will be to verify operating, maintenance and test 
procedures, to provide operational training and to test 


System upgrading [Ref. 8]. 


the Project Manager was faced with a complex probiem to 
solve. The problem contains the multiple measures of 
effectiveness of cost, schedule and future usefulness under 
conseguences of varying degrees of uncertainty. It is this 
problem which was used in Chapters IV and V to illustrate a 
decision analysis approach to project management trade-off 
situations. Chapter III contains a description of the 


methodology necessary to formulate and solve tne problen. 
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ITI. БЕСЕЕ5ТОН АЧА,Ү5%Ж5 METHODOLOGY 


TC A ee солы» A eee ee O ee O O > Ai ee O AM ee ee ee O A ee END PUE 


Man is often confronted with situations in which the 
consequences of any action he takes are not certain. Events 
may intervene which he can not control or predict with 
certainty. A large number of decisions under uncertainty 
are made by intuition [Ref. 9]. The intuitive decision 
process is accomplished in the decision-maker's ind. 
Because of this, there is no way to verify that this type of 
decision is the logical consequence of the Choices, 
information and preferences that were available to the 
decision-maker. For many problens, however, it is important 
that the decision-maker is able to show people why he 
arrived at a particular decision and also for then to be 
able to see what changes in factors surrounding that 
decision might have led to a different decision [Ref. 91. 
Another characteristic of the intuitive decision process is 
the human tendency to equate the quality of the decision 
with the quality of the outcome it produces. For example, 
consider a situation where an investor decides to buy some 
new stock. If he loses money, the tendency is to say that 
the investor made a bad decision; conversely, if he makes 
money, then he made a good decision. A good decision is a 
decision whicn maximizes: the probability of a good outcome; 
hence, making a good decision is no guarantee of a good 
outcome. The decision-maker has control of the decision. 


"He does nct have control of the outcome. 


КЕ” ритрове ОП (Песісісі алайузіс us to allow the 
decision-maker to make consistent good decisions and to 
formulate them in quantitative terns that can be conveyed 


from one person to another. Formal decision analysisisa 
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systematic process comprising the following steps: 
(1) structuring the problen, 
(2) assessing relative preferences for possible 
consequences, 
(3) evaluating the probabilities for uncertainties and 
(4) determining the best course of action from the 
information in the preceeding Steps [Ref. 10]. 
This process is an iterative process. First, a broad 
description of the problem with rough assessments of the 
preferences for the consequences and probabilities fcr the 
uncertainties is analyzed. On the basis of the first 
analysis, alternatives are added Or removed from 
consideration. The measurements are refined and the process 
is repeated until there is satisfaction with the results of 
the analysis [ Ref. 11]. 


The purpose of this chapter is to acquaint those 
unfamiliar with decision analysis with its theory and 
techniques to the extent necessary to formulate and solve 
the PF Land Eased Test Site problem introduced in Chapter 
II. This chapter is organized to explain the methodolcgy of 
decision analysis for each step іп the formal analysis. 
Before proceeding, it is necessary to explain certain terns 
and notations which are used throughout the remaining parts 


orf this thesis. 
А. CLARIFICATION OF TERMS AND NOTATIONS 


ШЕШ ter Ús "isk indifferent to", "is preferred to", 
Elottery" and "utility function", are widely used in the 
following sections of this thesis. Foc clarity, they need 
to be explained. The term "is indifferent to" is to be used 
to mean the same as the statement "the decision-maker 15 
indifferent to receiving either of the outcomes." The tern 


"A is preferred to B" is to be used to mean the same as the 
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statement "the decision-maker prefers A over B." 


The term "lottery" is defined as a gamble of some 


ж 
uncertain event E where the prize X is won if the event E 


occurs and the prize x is won if the event E does not 


ж 
occur. Let p represent the probability that Е occurs and 


ж 
let 1- p represent the probability that E does not occur. 


Notationally, the lottery я will be represented as 


ж ж 
«X ,p eX 2. 


The term "utility function" is defined asa function u 
which assigns a real value to every consequence a and Eb such 
that u(a) is larger than u(b) if ana only ii a is preferred 
Шо “р 3S[ Ref. 10]. The notation u(a) is expressed as the 
"utility of consequence a" and is represented by a real 


number. 


With the above terms clarified, the steps іп а formal 
decision analysis process can be explained. The first step 


in this process is structuring of the problem. 
B. PROBLEM STRUCTURE 


In structuring a problem in which eventS are uncertain, 
the options or alternatives are enumerated. Next, all the 
events that can possibly occur are specified. А5 а last 
step, the alternatives and uncertain events are arranged in 


chronological order. 
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A type of diagram known as a decision-flow diagram or 
"tree!" is a useful tool in decision analysis. It is a 
chronological arrangement of the alternatives which are 
controlled by the decision-maker and the events determined 
by chance [Ref. 11). To illustrate the construction of a 
decision-flow diagram, consider the following problem. А 
decision-maker is faced with two alternatives, I and II. 
Both alternatives involve a situation where the outcomes a 
or b are uncertain. If a occurs, then the decision-maker 
must decide between alternatives III and IV. Alternative 
III also involves an uncertain situation leading to either 


the outcome c or d. 


The decision-flow diagram is shown in Figure 3. Observe 
that the branching points or forks are of two types: 
decision forks and chance forks. A decision fork is 
designated by a small square and a chance fork by a small 
circle. There is additional information provided in the 


diagram which will be discussed in the secticns below. 


With the alternatives and uncertain events descrited by 
a decision-flow diagram, the next step in the decision 
analysis. process is the assessment of the relative 


preferences for the consequences. 


C. ESTABLISHMENT OF PREFERENCES 


The establishment of preferences for the consequences 
ШО Дес the; decision-maker with the basis for the rational 
choice between the alternatives. This depends upon the 
views and attitudes of the decision-maker. The consequences 
May encompass a number of factors or attributes such as 
cost, schedule and performance. These attributes might also 
be of an intangible nature such as. goodwill, morale and 


politics. 
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EXAMPLE DECISION-FLOW DIAGRAM 
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In this step in the decision analysis process, ап 
objective function is defined to indicate a measure for the 


preferences for the consequences. 


A general methodology for defining an objective function 
in decision analysis problens exists in the form of utility 
theory. Іп this thesis no attempt is made to develcp the 
theory in detail. It will be developed only to the extent 
necessary to formulate and solve the PF LETS problem. The 
development draws heavily upon work by Giauque [Ref. 12] and 
Keeney [Ref. 13]. 


Consequences may be described by a single attribute or a 
multiple set of attributes. Both situations are applicable 
to the analysis in this thesis and are separately presented 


below. 


In the case of a single attribute, an objective 
function, hereafter called a utility function, сап be 
defined which has the property that the maximum expected 
utility among the alternatives indicates the most preferred 
gor gon [ Ref. 12). 


A utility function with a single attribute сап be 
| x 
constructed in the following manner. Define X and x as 
the upper and lower limits over a range of possible 
consequences X, such that X 2X 2X For every possible 
1 1 


conseguence X „ define the utility u(X.) as the value p, 
1. i i 
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such that the decision-maker is indifferent to receiving X, 
і 


ж 
for certain and receiving the lottery <X ,p ,X >. The value 
1 ж 


ж 
of p, ranges from zero to one, where by convention, u(X ) 
i 


equals one and u (X. ) equals zero [ Ref. 12]. 


Once a set of points (X,,p.) have been established, 
rei 
a utility curve may be drawn. Figure 4 illustrates . three 


possible utility curves. A utility curve generally has two 
characteristics. It is smooth and the general shape of the 
curve is either convex, straight or concave as illustrated 
respectively by curves 1, 2 and 3 of Figure 4. Any break in 


the curve would indicate either an inconsistency in the 


ж 


choices for p, in the lottery <X ,„p.,X > used io assess the 
1 1. ж 


points of the curve, or a quantum jump in preference for a 


small change in X . A convex curve indicates a risk averse 
! di 


behavior (Ref. 111. That 15, the decision-maker is more 


inclined to take a consequence known for sure than to take a 
gamble with the same expected value. A concave curve 
indicates that the decision-maker is risk seeking. He 15 
more inclined to take the gamble than to take the known 
consequence. A straight line indicates that the 
decision-maker acts on the expected value of the conseguence 


[Ref. 11]. He is neither risk averse nor risk seeking. 
Once the utility curve is established for a single 


attribute consequence, a value from one to zero is assigned 


to each consequence corresponding to the point on the curve. 
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А higher value for a consequence indicates greater 
preference for that consequence than for a consequence with 
a lower utility value. Reference 14 contains detailed 


treatment of utility functions with single attributes. 


EXAMPLES OF UTILITY CURVES 


е | EL 
1 S 
° ә С 
Utility y 
Za 
P. 
1 / 
2 ж 
0.5 a 
2 
/ 
” 
/ 
A 3 
/ 
Z 
и ее аванс 
ж 
X X 


Consequence X. 
1 


FIGURE 4 
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The basic concept of the construction of a utility 
fünction with a single attribute described’ above can Бе 
generalized to the case where many attributes must be 
considered. However, the above assessment Scheme is 
impractical. TIN SOME O OMA NADO 1N ESO must be fassessed- 
эт humans Land it difficult to think іп terms of 
nultiple attributes. In decision problems under 


uncertainty, many people when faced with situations where 
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more than one attribute is relevant, tend to pick the one 
attribute judged most important to them and then make the 
decision on that factor alone [Ref. 10]. 


There are procedures for decomposing a multiple 


attributed utility function into combinations of 
unidemensional functions. Conditions required for 
decomposition include the properties of utility 
independence, pairwise preferential independence and 


pairwise marginality. These are described below. 


Keeney [Ref. 13] shows that a pultiattributed 
utility function can be expressed in one of two forms, 
additive or multiplicative, dependent on which o£ the 
properties of utility independence, pairwise preferential 
independence or pairwise marginality hold. 88 20 Ea ay 
function of multiple attributes can be expressed in these 
forms, then the task of defining the utility functicn is 


much easier. Suppose X = C ) descrites а 
n 
conseguence where u(X) denotes the utility of the 


consequence X. Utility independence is defined in the 


MONO Wtng manner, тет х = (х.х ,Х. vss, X te The 
1 1 1-1 1+1 n 


attribute x 15 utility independent of x,_ if the 

i i | 

decision-maker's relative preference for x , with x, held 
i Я. 


fixed, is the same regardless of the actual value of x,. 
! i 


chosen. Order one mutual utility independence is defined to 
mean that x, is utility independent of x, for all i. If 
i i | 


order one mutual utility independence holds then u(X) can be 


expressed in the quasi-additive form 


1 
E 
ғ" 
” 
u 
+ 


n n 
ши ек. | Los 2 b ООСО ОАЗЕ 
1 n 1511 1 Еа 92 


За 





Pairwise preferential independence is said to hold 
if the trade-offs one is willing to make between attributes 


taken two at a time, are not dependent on the values of the 


remaining attributes. Let ХӮ = = 
17 
(х se. X, "24 e ie X, ex. oo. Х )v and let X. es be a 
1 4-1 іні 3- 08 зе: n 17 
particular value fron X, _. The attributes Xx., x, are 
1) 1 J 
pairwise preferentially independent of X, if one's 
13 
preference order for the consequences (x, х, Х 2) wath 
1 J 1) 
x | held fixed, does not depend on the particular value 
1) 


x, _ [Re£f. 13]. 
ij 


If for any pair of attributes x, and x , the lcttery 
- J 


0 0 
2х 00.53 (х ,Х )> is indifferent to the lcttery 
i i j 


< (хх ),0.5, (х ‚х )> then pairwise marginality is said to 
l 1 5 E 7 


mold [Refs 127. 


With the ideas of utility independence and pairwise 


preferential independence presented, Keeney's results can be 


more precisely stated [Ref. 13]. Let X = CN LENTES: ) be as 
n 
previously defined, with n 2 3. If, for some X., X, and Хх 
i i j 
are | pairwise preferentially independent of 
DES .,,x Xx —...,X )» for'allj * i ant x, 
1 и у= үе n 1 
is utility independent of x, , then either 
i 
Т | 
Ч М (с) . (1) 
у=] EI 
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ог 
n 
1+ K u(X) = fy tt t Kk u, (x,) ] (2) 
i= i ji 


where u and u, аге utility functions scaled from zero to 
i 
one, the k, are scaling constants with O < k, < 1 and K > -1 
i 1% 
is a non-zero scaling constant. Equation (1) is the 


additive form and equation (2) is the multiplicative form. 


Given that the conditions of Keeney*s Theorem hold, 
he provides a property required to show whether the function 
is additive (1) ог multiplicative (2). He shows that if 
pairwise  marginality holds then the function must be 
additive; otherwise, it is multiplicative. Table I 
summarizes the properties necessary for each simplification 
ШЕСЕ. 121. 


TABLE I 
UTIILITY SF UNCTIONSSINPLIFICAMION 


Simplification Properties 














Ist order] Pairwise 









Palryvise 






utility preferential 


indep. indep. marginality 
Quasiadditive forn 
Multiplicative form 


Additive form 


Referring to Figure 3, there are three attributes 
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E. E and ХА which describe each outcome of the tree. For 


illustration, the following utility function might be used : 
Bx X ,X - u(x * u(x + u(x = 14. 

( и Хо" 3! ( 1) ( >) ( 3! i 
This utility function, in the additive form, maps the 


consequences ХЕ 3d and > into a scalar value indicated by 


и, where i = 1,...,8, at each branch-tip of the tree. 
i 


D. JUDGHENTAL PROBABILITIES 


The decision-flow diagram is one of the decision 
analysis methods used in structuring a problem. Utility 
functions can be used for the assignment of preferences for 
the consequences of the outcomes at each tip of the tree. 
What remains to complete the information included cn the 
decision-flow diagram is the assignüent of the judgnenta 
probabilities at the chance forks representing the uncertain 
events. This is the third step in the decision analysis 


process. 


Raiffa [Ref. 11] addresses the question of whether the 
decision-maker's hunches or vague impressions should Бе 
calibrated, and if so, how this should enter into the formal 
decision analysis process. He argues that 11 жа 
decision-maker wishes to act consistently, then he ought to 
assign values to judgmental probabilities such that the sum 
of the probabilities of an event occurring and not occurring 
equals спе. This judgmental probability assessment for an 
event shculd not depend on the outcomes. He points out that 
judgmental probabilities satisfy the usual rules of 
probability theory and can be used in the same manner as 


objective probabilities. 
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Judgmental probabilities are used as a measure of the 
decision-maker's beliefs concerning the uncertainty of an 
event cccurring, provided that these beliefs are 
consistently applied to every uncertain event in the 
analysis. They are assigned to each chance fork of the 


tree. In Figure 3, they are represented as Pie ur Pos 


1- 1- апа 1-р . With this information the 
Pot Ра” Poe Pu P , 


final step in any iteration of the decision analysis process 


is to determine the recommended course of action. 
Е. RECOMMENDED COURSE OF ACTION 


Determination of the recommended course of action 
involves a sequence of calculations calleä by Raiffa [Ref. 
11] the "averaging out and folding back" procedure. This 
procedure is often reierred to as the process of backwards 
induction in the theory of dynamic programming [Ref. 11]. 
The procedure starts at the tips of the tree and consists of 
computing the expected utility of each chance fork and tne 
selection of the greatest utility at each decision fork. 
The process is repeated for each level of the tree until the 
Starting decision fork is reached. The alternative with the 
greatest expected utility is selected as the recommended 
course of action. The selection of the maximum expected 
utility is an appropriate means of determining actions 
consistent with the decision-maker's attitudes and opinions 
[ Ref.. 12]. This point is presented and developed in such 
Sources as Schlaifer Ше] Жс Мер ггасо Raltfal and 
Echlaifer [Ref. 1]. 


To illustrate the “averaging out and folding back" 


process, the information contained in Figure 3 15 used. 
Starting at the chance fork labeled DE tiewexpected 
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utility is computed as 


u + u 1- = E 
1 2 2 : Ра; 
Moving backwards in the tree, the next fork encountered is a 


decision fork, labeled |2]. The value of ER or er 


whichever is greater, is selected. For illustration, Бы 15 
selected. Continuing backwards through the tree, a chance 


fork, labeled EN is encountered. At this point, the 
expected utility of the chance fork is computed as 

E + п a = E. 

1 Р 4 : Р) 2 


Alternative I has now been reached and the expected utility 


of this alternative is D In similar fashion, the exrected 


utility cf alternative II is computed. The results are 
compared and the alternative with the greatest expected 


utility is selected as the recommended course of action. 
F. SUMMARY OF CHAPTER 


On most occasions, people make decisions intuitively and 
more or less inconsistently. There are occasions when the 
decision must be made in а геавопей, deliberate manner. 
Decision analysis methodology was introduced to provide 
this. In the systematic process of decisicn analysis, the 
decision-maker starts by structuring the anatomy of his 
problen in a decision-flow diagram that depicts the 
chronological interactions between his alternatives at any 
stage and the events which are controlled by uncertainty. He 
scales his preferences for the consequences at the tips of 
the decision tree in terms of utility values and scales his 
judgments abcut uncertain events in terms of probability 


assignments at the chance forks ina consistent manner. 
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Finally, he selects his best strategy for action by the 
process of "averaging out and folding back." 


Most "real life" problems are complex. Trees exhiriting 
the structure of these problems can be so complex as tc make 
adetailed analysis of the alternatives impractical. There 
is an "art" to analyzing real problems as descriked by 
Raiffa [Ref. 11]. An iterative process is used. Initially, 
alternatives and measurements are specified іп а rough 
manner. Frequently, sone of the decision branches will turn 
out to be nonoptimal and can be eliminated from the tree. 
If the decision branch is close to the base cf the tree, a 
sizable portion of the tree can be eliminated. After 
elimination, further effort can be put into refining the 
description of the remainder cf the tree. This iterative 
process is repeated until the decision can be satisfactorily 


made. 


The necessary decision analysis methodology has now been 
provided. Formulation and solution of the PF Land Based 
Test Site problems are presented in the following two 


chapters. 
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IV. PF LBIS PROBLEM FORMULATION 


= «құс “ео cee ee es ee agemus anms често eee ee eee eee O E Gm 


Chapter II presented the background of the PF Land Based 
Test Sites and developed the trade-off problem for the 
Program Manager as to what would be the necessary mix of PF 
subsystems which would be installed at the LBTS to insure 
adequate testing for the DSARC III review process. The 
Project Manager has to make sure the testing of the 
subsystems installed at the LBTS provide a valid estimate of 
the expected systen operational effectiveness and 


Suitability. 


The Development Concept Paper [Ref. 7] lists in the Risk 
Watch List the PF subsystems which are considered to have 
some degree of development risk. Among these items are the 
Mark 92/2 fire control system, the AN/SPS-49 radar and the 
LM 2500 gas turbine engine. Individual Technical and 
Operational Evaluation are to be conducted on ive subsystems 
listed (Ref. 5], but some seemed candidates for the Land 


Based Test Sites including the fire control system, radar 


and engine. In this chapter, the options for these three 
subsystens are examined more closely using the tools 
developed in Chapter Ill. This examination comprises an 


initial analysis proceeding in the decision analysis steps 
previously described. Considerable data for this analysis 
was provided the author by the PF Test and Evaluation 
Manager and the Assistant Test and Evaluation Manager during 
interviews with them at the PF Project AR > b y 


correspondence and by telephone interview. 


The first step in the decision analysis process is to 


Structure the problem in terms of a decision tree The 
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formulation of the problem structure is described below. 
A. PROBLEM STRUCTURE 


In testing of the Mark 92/2 FCS, SPS-49 radar and LM 
2500 gas turbine engine, the Project Manager is concerned 
with the installation and testing costs, the schedule, the 
usefulness of the subsystems to the PF Project at the LBTS 
in the future, and the uncertainty of passing DSARC III. 


His options for all three subsystems, in general, are: 
(1) install actual or simulated equipment at the LETS, 


(2) conduct at-sea testing of each subsystem during 


Operatıonal Evaluations, 
(3) monitor tests at the vendor's activity or at sea or 
(4) conduct no further testing. 


To structure the BES problem, the options or 
alternatives are enumerated. Next, all the events that can 
affect the installation and testing of the subsystens EC 
listed and arranged in chronological order. Dastlym a 


decision tree is constructed. 


An intuitive process, in considering the options, was 
suggested by the PF Test and Evaluation Manager. This 
process is shown as a logic-flow diagram in Figure 5. TO 
enumerate the alternatives, the options displayed in the 


"ULBTS" boxes of the logic-flow diagram are examined. 
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INTUITIVE LOGIC-FLOW DIAGRAN 
PF LBTS PROBLEM 


Is system critical? 


Yes 
No 


Place on Risk Watch List 


Yes 
Is interface of 
complicated nature? 


No 


Does complex nature 
Nor шап мсесс Е of 
installation at LBTS? 


Yes No 


Install and test 
Simulated equipment 
NETS 





Install and test 
»actual equipment 
ан LBIS. 





Is system a ma Jor 
PF component: 
Yes 


No 


Yes 


Is system bein 
installed in other 
Ship programs? 






| No} 
PF Project | 
шепа ores No testing Conduct 
atte sea requined. at sea 
testing. testing. 
FIGURE 5 
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Basically, there are four alternatives considered. 
The first alternative is to install the actual subsysten in 
its entirety. The second alternative is to install a 
completely simulated subsystem. The third alternative is to 
install a partially simulated subsysten. The fourth 
alternative is not to install the subsystem at the LBTS. 


The above alternatives apply for the installation 
and testing problem of the Mark 92/2 FCS. They dc not 
entirely apply for the cases of the SPS-49 radar or LM 2500 
gas turbine. The alternative of partially simulating some 
components for the SPS-49 radar system was not considered. 
Шопга е Sustias feasible Фо sinmúlatiel the entire “radar 
system as to simulate some components such as the antenna 
and transmitter. Therefore, three alternatives were  cnosen 
for the AN/SPS-49 radar  proolenm. They are: install tne 
actual equipment of the radar, simulate the radar system or 
not install the radar at the L3TS. 


The alternative of partially Simulating the gas 
turbine engine was not considered because it does not make 
sense to do so. The alternative of not ‘installing the 
engine at the Propulsion LBTS was also not considered since 
the engines must be represented if there is to be a 
Propulsion LETS. The assumption is made that the Propulsion 
System LBTS is desirable and necessary for the develcpment 
of the Patrol Frigate. For the LM 2500 gas turbine engine, 
two alternatives are considered. They are to install the 


actual engine or to Simulate the engine at the LBTS. 


With the alternatives enumerated for the 


installation and testing of the three PF Subsystems at the 
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LBTS, the events which affect them are specified and 
arranged in time sequence. 


The uncertainty of passing DSARC III scheduled for 
May 1975 is the primary concern of the PF Project Officer. 
The DSARC III review determines whether the PF develcpment 
risks have been minimized so that the PF Acquisition Project 
May progress into the production phase. It is assumed that 
the Patrol Frigate is a necessary addition to the military 
posture for National Defense. If the DSARC III review does 
not consider the technical and project risks pe be 
minimized, DSARC III will be rescheduled after all the 
deficiencies have been corrected. It will be at this point 
where the decision tree structure will be cut-off in the 


initial analysis of this thesis. 


Ihere are two events besides the DSARC III review 
which must be considered in structuring the decision tree. 
The first event is the LBTS Certification tentatively 
planned for January 1975. The second event is the 
Operational Assist which will ре conducted by Commander 
Operational Test and Evaluation Force (COMOPTEVFOR) during 
the period February to Hay 1975 [Ref. 17]. 


LETS Certification will consist of a series of tests 
conducted by the) GBTS contractor to verify that the Combat 
System subsystems and Propulsion System subsystems have been 
successfully integrated, are operational and are ready to be 
used for other demonstrations at their respective  LBTS. 
Certification will be approved by the PF Project Manager. 
If the LBTS fails Certification, additional time and money 
will be used to correct the deficiencies noted in order to 


bass GCertafticgacion. I£ fundsi are not sufficient to correct 


42 





the deficiencies, the LBTS will have to be cancelled. 


Another decision event OCCULS prior to 
Certification. If there is enough time to do Certification 
and Operational Assist, then Certification will commence as 
Scheduled. If there is not enough time to do both, then the 
certification process will not be formally done and the 


Operational Assist vill commence as scheduled [Ref. 17). 


An Operational Assist project is assigned by the 
Chief of Naval Operations (CNO) in response to a favorable 
program initiation decision by the Secretary of Defense. 
For the case of the Patrol Frigate, it was assigned by nef. 
4. The major purpose of an Operational Assist is to 
establish confidence in the program worth and readiness for 


the commitment of resources for full-scale development [Ref. 


18 1. А decision event occurs if the Operational Assist is 
determined not satisfactory by the COMOPTEVFOR 
representative. At this point in time, it must be 
determined whether to correct the itens which are 


unsatisfactory or to cancel the LBTS. Table II summarizes 


these events and their consequences in time sequence. 
With the alternatives and the {uncertain events 


arranged in chronological order, a decision tree is 


constructed to complete the structuring of the LBTS prcblen. 
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TABLE II 


UNCERTAIN EVENTS OF LBTS 


EVENT CONSEQUENCE TIME-FRAME 







i. Time available If time available, 
do both; else, 


do Op Assist. 


Fall 1974 
for certification 


and Op Assist 






2. Certification IÉ 


passed, do Op Assist. 


Pass or fail. January 1975 







If failed, do over 


or cancel LBTS. 






3. Op Assist If 
passed, do DSARC III. 


If failed, do over 


Pass or fail. DR LS 






Or cancei LBTS. 


4. DSARC III Pass or fail. If May 1975 
passed, analysis done. 
If failed, do over. 


-— auum oe (rar =й» uum 


Figure 6 depicts-a typical branch of the decision 
tree for the alternatives of actual installation, partial 
installation or simulation of the subsystems at the LBTS. 
"The first chance fork in the branch represents the 
uncertainty of the time available for Certification and 
Operational Assist. Only one alternative is selected at 
this point. If there is time avallable, then the 
alternative is to do both Certification and Operational 


Assist. If there is no time available for both tests, then 
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the alternative is to do Operational Assist only. Тһе 
second chance fork represents the uncertainty of 
Certification. If it is passed, then the Operational Assist 
is conducted. If it is failed, then a decision is made to 
cancel the LBTS or do the Certification over. The third 
chance fork of the branch represents Operational Assist 
testing. If the Operational Assist is passed, then DSARC 
III commences as scheduled. If the Operational Assist is 
failed, then a decision is made to cancel the LBIS or do the 
Operational Assist over at a later date. The final chance 
fork is the DSARC III review. f it is passed, the most 
preferred corsequence is achieved. If it is failed, then a 
decision is made to either cancel the РЕ Project or to 
reschedule DSARC 111 at a later date. As was pointed out 
above, the assumption is that the Patrol Frigate is 
essential to the National Defense posture, so that, in the 
analysis, the "do over" branch will be weighted more 


favorably than the "cancel" branch. 


Figure 7 depicts the "no installation " branch of 
the decision tree for the Mark 92/2 FCS and SPS-49 radar. 
There is cne chance fork representing DSARC III in which it 


is either passed or failed. 


The structuring of the Land Based Test Site problem 
is completed. The next step in the decision analysis 
process is to evaluate the consequences of the decision tree 
in terms of utility numbers. This process is described 
below where a set of utility functions for one individual 
concerned with the problem is developed to illustrate the 


method and use of decision analysis. 
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INSTALL EQUIPMENT BRANCHES 
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Op Assist © Cancel 
Fail 
Pass Do ovel | 
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Fail | 
Do ove | 
Тіпе 
DSARC | 
| ‘III 
o time Pass 
Cancel | ———— ——^ 
Fail 
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FIGURE 6 
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TYPICAL LBTS DECISION TREE 
INSTALL NO EQUIPMENT BRANCH 





FIGURE 7 


B. ASSIGNMENT OF UTILITY NUMBERS 


In order to provide a basis for the rational choice 
between the alternatives, utility functions are established 
to indicate a measure for the preferences of the 
conseguences. The consequences for the LETS proklem are 
described by the attributes of cost, schedule and future 
usefulness. A fourth attribute, performance, was a 
possibility. It is assumed that all three subsystems will 
have to meet some previously specified performance level; 
therefore, a performance attribute was not considered for 


this initial analysis. 


The attributes which describe the consequences are 
tangible and are neasured TN ELO NS PO O 1 a LS L Or LME COSL, 
months for schedule and scalar values for usefulness. The 
scalar value for usefulness ranges from zero to three. The 
value zero is assigned for a subsystem which is not 


installed at the LBTS. The value one is ssigned to a 


a 
simulated subsysten. The value two is assıgned to a 
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partially simulated subsystem and the value three ls 
assigned to the installation of the actual subsystem. Table 
III lists the attribute ranges determined from interviews 
with the PF Test and Evaluation Manager and the Assistant 
Test and Evaluation Manager [Refs. 17,19]. The values for 
cost and schedule have been generalized reflecting an order 


of magnitude only. 
TABLE III 


ATTRIBUTE RANGES 


COST USEFULNESS SCHEDULE 


(million) (scalar) (month) 


1. AN/SPS-49 





ninimum 

maximum 4. 
2. MK 92/2 FCS 

minimum 1. 

maximum 6. 
8, ІМ 2500 67 

minimum 10.0 

maximum 16.0 


Since a consequence for the LBTS problem is represented 
by multiple attributes, the utility function will ре 
described, in some form, by a combination of the individual 
attribute utility functions. Once they are established, the 
utility function might be expressed in the additive ог 
Multiplicative forms exhibited in Chapter III dependent on 
the properties of the attributes. EStablishment of the 


individual attribute utility functions is first discussed. 
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A discrete utility function consisting of four 
points was established for the usefulness attribute in the 
analysis of all three subsystems. A straight line utility 
function for the schedule attribute for the SPS-49 radar 
decision tree was derived and a family of four utility 
curves was established for the schedule attribute describing 
the consequences of the fire control and gas turbine 
decision trees. À Straight line utility function 
representing the cost attribute was derived for all three 
Subsystem decision trees. Table IV summarizes the results. 
The method for arriving at these utility functions is 


discussed in the paragraphs below. 


The attribute of future usefulness consists of four 
integers, zero, one, two and three. The utility of the 
value zero is Zero since it is least preferred and the 
utility of the value three iS one since it is most 
preferred. To assess the utility of the values one and two, 
the PF Assistant Test and Evaluation Manager was asked the 


following questions [Ref. 17]: 

"If you are indifferent between the future 
usefulness value one and the lottery рр what do you 
feel the probability p. of getting a future usefulness value 
three is?" 

"Tf you are indifferent between the future 
usefulness value two and the lottery ЕР, о; what do you 
feel the probability = of getting a future usefulness value 


three is?" 
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тһе reply to these questions was that ра would be 
0.5 and ра would be 0.85. With this information, the 
utility function for the attribute future usefulness was 


evaluated. Given four discrete values, 0, 1, 2 and 3, with 
лам = 0 and EC = 1, then , 


u (1) = 0.5 u (3) + 0.5 u, (0) = 0.5 


and 


o 0.85 nn) +0.15 uM CO) = 0.85. 


A utility measure for the schedule attribute for the 
three PF subsystems will not have similar worth. AS was 
disclosed in conversations with the PF Project Assistant 
Test and Evaluation Manager [Ref. 20], the schedule for the 
AN/SPS-49 radar is not critical. In assessing a utility 
function for this subsystem, he would be indifferent to 
gambles with respect to the schedule. He also indicated for 
the Hark 92/2 FCS and gas turbine engine, their schedules 
are very critical. If he is afforded a choice between a 
gamble consisting of uncertain schedules and a schedule 
which was kncun, then he would take the known Schedule in 
every case, indicating a risk averse behavior. He added 
that as long as the installation and test schedule were met 
for DSARC. III, he would evaluate the utility of any schedule 


less than the maxımum-as one. 


The ranges for the maximum and minimum schedule for 
the radar are 1.5 and 4.0 months. Since the preferences for 
schedule uncertainties for the SPS-49 radar are not risk 
averse, a Straight-line utility function can be used to 


express the expected value for the installation and testing 








schedule. The | straight line connects the point 
Be u U 0,50) with the point Сора Uis Since 


the function represents the expected value of the  SPS-49 
schedule, the equation of the utility function cf the 
attribute schedule, ще “111 Бе 


X 


1. = : 
> B» 5 + [1 ci 4.0 


ЦЕ = 2.5 44 (X ). 
2. 2 


Непсе, 


= (4 > 2,58 
ux.) ( х.) / 


The utility function for the attribute schedule of 
the SPS-49 radar, is represented in the following graph: 


AN/SPS-49 Radar 


Utility 0.5 





Schedule (months) 


The LM 2500 gas turbine engine schedule ranges from 
ten to sixteen months. The Mark 92/2 FCS schedule ranges 


from 1.5 to six months. Because the schedule 15 zeritical 
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for these two subsystems, there will be risk adversion 
towards any gambles of uncertain schedules. A utility value 
of one will be assigned to any schedule that is less than 
the planned schedule for DSARC III. With this information, 
the utility function curve for the fire control system would 
be similar to one of the curves in the following graph. 
Four utility values, as indicated on the grapn, will be used 


to determine the sensitivity of the results. 


MK 92/2 FCS 


Utility 0.5 


0.0 
Schedule (months) 
The curve for the utility function of the gas 


turbine schedule would similarly look like one of the curves 


in the next graph. 


Gas Turbine Engine 


T Т 


Utilit 
Yo 


10 16 


Schedule (months) 
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Values for the utility function for schedule for the engine 
and fire control system will be obtained from each of the 
four curves of the two graphs above and used in the solution 
of the decision trees to determine the sensitivity of the 
results. If there is a significant difference in the 
solution results between these values, then further research 


in the develcpment of the utility function will be required. 


The evaluation of the utility function for the 
attribute cost was next considered. The installation and 
testing costs for the SPS-49 radar range from 1.5 to 3.5 
million dollars. Тпе ІМ 2500 gas turbine engine costs range 
from three to fifteen million dollars. Тһе Mark 92/2 FCS 
costs range from 7.5 to 18.5 million dollars. The minimum 
costs represent the best costs for installation and testing, 
if all events are passed on schedule. The maximum costs 
represent the worst costs as if all events are failed and if 


no more funds are available to correct deficiencies. 


It seemed plausible that one utility function for 
the three subsystems could be derived where the costs range 
from zero to 18.5 million dollars. From discussions with 
the Assistant Test and Evaluation Manager [Ref. 20], this 
seems to be a valid assumption. ' He is willing to take 
gambles on the basis of the expected cost values over the 
entire range for the three subsystems under consideration. 
This is justified because a vast range of assets 15 
considered. A decision-maker, considering the vast amounts 
of funds available, would not be necessarily risk averse. 
It is realized that the cost decisions for each subsysten 
would be made over a much smaller range as indicated for 


each PF subsystem. 


The utility function for costs was derived as a 


straight line connecting the point a оон) =0) 
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with the point RS ge E The equation cf the 


utility function of cost, CE is 


x = u (X 0 *[1- u (x 1855 
Q7 24 0) [ 363) ] 
3,5 - 18:56” (Хх. )% 
З З 
Непсе, 
u (x = (18.5 - x 13,5 
5 3! ( 3! / 


The utility function for the cost attribute is 
represented in the fclloving graph. 


Utility 


Cost (million) 


Hith the individual attribute utility functions 
specified, the utility functions for the fire control, radar 
and gas turbine engine conseguences can be represented as a 
combination cf the three individual utility functions. This 
utility function may be expressed in the multiplicative or 
additive form dependent upon which of the attribute 
properties of utility independence, pairwise preferential 
independence or pairwise marginälity hold. The next section 


is devoted to the verification of which of these attribute 
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properties hold. 


TABLE IV 


INDIVIDUAL UTILITY FUNCTIONS 


Attribute Applicable to Utility Function 
1. Future usefulness: А11 ши - 0.0 
.u (1) = 0.5 
я: ) 
(2) с 0.85 
1 
u (3) = 1.0 
1 ) 
2. Schedule: SPS-49 radar | = И 8 
МК 22 ЕС5 Family of curves 
Gas Turbine Family of curves 
3. Cost: All ue (x = (18.5-x ac 
3 | 3! ( 27 


2. Attribute Properties 


If the attribute properties of order one utility 
independence and pairwise preferential independence hold for 
the attributes cost, schedule and usefulness, then the 
utility function can be expressed in the multiplicative 
form. Further, if the property of pairwise marginality also 
holds, then the utility function can be expressed in the 


additive form. These forus are shown in Chapter III. 


Verification or utility independence for the 
attributes cost, schedule and future usefulness of the Mark 


272 WES problem 15 first considered. rO verily utility 


л 
л 








independence, consider the attribute cost of the Mark 92/2 
FCS installation at the Combat System LBTS. The minimum and 
Maximum costs were determined to be 7.5 million dollars and 
18.5 million dollars. The following question is asked [Ref. 
15]. 


"Suppose the values for future usefulness and 
Schedule for the fire control system are specified at some 
given quantity, say, three and 1.5 months. Now, consider 
the gamble 7.5 million dollars with a probability of p and 
18.5 million dollars with a probability of 1-p. Determine a 
dollar amount X such that if you had to choose either the 
gamble or the X million dollars for sure, you would be 
indifferent. Now, suppose I dive you a different set of 
values for the other attributes, say, two and fifteen 
months, and ask you to assess X again. Does the value of X 


million dollars change?" 


If the value of X million dollars does not change no matter 
what values are given for the usefulness and schedule 
attributes, and if this is true for all gambles on the cost 
attribute, then the cost attribute is utility independent of 
the other attributes. Utility independence of the 
usefulness and schedule attributes can similarly be verified 
[Ref. 15]. If all of the attributes are utility independent 
of the other attributes, then order one utility independence 


is established. 


Ic verify pairwise preferential independence, 
consider the attributes of cost and future usefulness for 
the gas turbine problem. The followiny dialogue is held 
[Ref. 15]. 


"Consider the consequence I, involving a four 


milton дот Так installation and testing cost for the gas 
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turbine engine, no future usefulness and an eleven month 
schedule. Now, determine a dollar figure X such that 
consequence II defined as; X million dollars, a maximum 
future usefulness value of three and an eleven month 
Schedule, is exactly as attractive as consequence I. now, 
let's change the value of the schedule attribute. Does che 


value of X miliion dollars change?" 


If the value of X million dollars does not change fcr all 
values of the schedule attribute, and if this is true for 
all trade-offs between cost and usefulness, then these two 
attributes are pairwise preferentially independent of the 
schedule attribute. Pairwise preferential independence 
between the cther pairs of attributes is determined in a 


Similar manner [Ref. 15]. 


То verify pairwise marginality, a convenient test is 
the pairwise marginality test [ Ref. 15]: Ав ап 
illustration, the attributes of cost and schedule for the 


5р5-49 radar consequences are considered. Let lottery LA 
yield 3.5 million dollars and a schedule of 1.5 months with 


a probability of 0.5 and 3.5 million dollars and а schedule 
of four months with a probability of 0.5. Let lottery In 


yield 3.5 million dollars and a schedule of four months with 


a probability of 0.5 and 1.5 million dollars and a schedule 
of 125 months with a probability of 0.5. If the 


decision-maker is indifferent between LA and ро; Елеп 
pairwise marginality holds between cost and schedule. 


Similarly, the pairwise marginality tests are made between 


all pairs of the attributes. 


With the information gathered from interviews at the 
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Patrol Frigate Project Office, it was not unreasonable for 
the authcr tc verify the properties of utility independence, 
pairwise preferential independence and pairwise marginality. 
It was determined that the properties of order one utility 
independence and pairwise preferential independence hold for 
the attributes. Pairwise marginality does not hold. This 
result was the same for the three subsystems under analysis. 
By Keeney's Theorem (Ref. 13], the utility functions fcr all 
three subsystem problems can be represented in the 


multiplicative form 
1 + K u( Хх _) i 1 Kk (x.) ] 
* ПКК, ш + u. (x К 
123 ши ET 
where MC DE represents the attributes of future 


usefulness, schedule and cost respectively. The utility 


function of a single attribute x, is represented by и (х ). 
i т 


Іп the assignment of utility numbers to 
consequences, the individual attribute utility functions 
have been determined and the utility function, which is a 
combination of the individual utility functions, can be 
expressed in the multiplicative form. What remains is to 


determine the scaling constants K, S КА апа ES So that the 


multiplicative utility function can Бе expressed in 
numerical terms. This determination is made in the section 


below. 


3. Assessment of Utility Function Parameters 


—— == шыты A AAN dumb A A A clo cn come emma == AA де реч Li a 


À procedure for evaluating the scaling constants, 
when the utility function is in the multiplicative form, 15 
found іп REE. 13 and Ref. 15. Тһе basic idea is to define 


four linearly independent equations, since there are four 
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unknowns, К, Ee My and са to be evaluated [Ref. 15]. The 
following scheme is a convenient method for obtaining the 


linearly independent equations. 


Consider the outcones described by three attributes 
x'a As a first step, the decision-maker is asked to rank 
i 
the attributes in order of preference, For illustration, he 
ranks the attributes x, such that x, > E > Pr The utility 
i 


values m, will next be determined from five outcomes. Let 
i 


outcome a represent the best possible set of attributes 


ж ж ж 
(a,b ,c ). Let outcome Хх represent the worst possible set 


ж ж ж 
ОЁ attributes (a,b ,Cc,)- Then u(a ,b,c) = 1 and 


ula »b .2,) = 0. Let outcome x, represent the set of 
attributes where the worst value for the least preferred 


x 
attribute, С г replaces c in the attribute set of outcome 


ж ж 
Х such that X - (a,b ,c). Similarly, let outccme X 
zi m * | 3 


represent the set of attributes where the worst value for 


ж 
the next least preferred attribute, bie replaces b , such 


ж ж | 
that ue = (a bc ye Using the same ccnventions, let 


ж ж 
outcome X = (a ,b ,C ). 
$y е. 


5/8) 





To assess the utilities of the outcomes X , Be and 
2 


X the decision-maker is asked to determine the 


probabilities of the gambles in the following situations 


where he is indifferent between: 


a) X for sure and the gamble «X XD 
(a) 2, 4 21,252. , 
(b) X. for sure and the gamble Pte and 


c) X for sure and the gamble <X К >, 
(с) zn g re, 5 


Cnce the probabilities p, p_ and Еһ are assessed, 


2 3 
then 
u (X - u (X + (1- 1(Х 
(X) р Ж.) ( Р.) X): 
u(X = u (X % (1- u(X and 
(X) P. (X) ( Р) (X) 
u (X = u (X + (1- u(X ). 
(X 0) P (X) ( 2%, (X) 
Since u - 1 апа u(x.) = 0, it follows that 
u (X - = m 
( 2! Р, >" 
u (X = = m апа 
к) ру = а, 
u (X = =m. 
( i P, " 


The utility values nr Д2 апа us can be verified for 


consistency by asking the decision-maker to assess other 
gambles consisting of various combinations of the five 
outcomes. If the decision-maker has been consistent, then 


the utility values derived should be similar to the 
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previously derived utility values T ще апа Dn If they 
are not similar values, the decision-maker is asked to 


re-assess all gambles until some consistency is achieved. 


Table V illustrates the above steps. 


TABLE V 


ILIUSTRATION OF SCALING CONSTANT EVALUATION 


OUTCOME ATTRIBUTES UTILITY OF OUTCOME 


X 
1 





Once the utility value m for each of tne. tive 
J 
outcomes are defined, the scaling constant K сап be 


determined frou four lineariy independent equations.  Keeney 
[Ref. 13] shows the following relationship holds for the 
multiplicative form: 


3 
1 K= 1 Kk 3 
кке Тр (+ кк.) (3) 


апа, general by [ Ref. 13]; 
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rear d (1 + Kk.) / (+ КК). (4) 
2 1= 1 2 


Equation (3) and equation (4) for j = 1,...,3 are used as 


the four linearly independent equations. From (3) and (4), 


3 
1* K- 1 * Kk 
pl] к 


= (1 + nK) (1 * Kk ). (5) 
J J 
( 1 K) 3 Е (1 K) (1 Kk ) 
* - + m + 
je I j 
3 3 
= H (1 + m K) 11 (1 + Kk ). 
js J gu J 
Then from (3), 
(1 + k)3 : (1 K) (1 K) 
- + m + 
14 j 
and 
аз юғ- 7р ак). (6) 
+ = m Е 
321 j 


Equation (6) is a cubic equation in the form, 
K (aK? * bK жс) - 0. 


Thus, one of the values of the cubic equaticn is zero. The 
other twc values are: 
1/2 
ЕЕ (о2- lac) Урак 

Keeney (Ref. 13] proves that the range for the values of K, 
will be K > -1 where K £ 0. Therefore, that value of K 
which is non-zero and greater than -1 is chosen. Giauque 
[Ref. 12] proves that there is always one and only one such 


value. 
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Once the constant K is known, the remaining scaling 


constants КА >> and = are derived from equation (5), 
1+ K = (1 + ик) (1 + кк); 
2 2 


Непсе, 


k =(1-m)/ (1 + mn K). (7) 
J 3 J 


Appendices A, B and C outline the steps and 
computations made to determine the scaling constants for the 
fire control system, the SPS-49 radar and the gas turbine 
engine problem utility functions. The scaling constants are 
listed in Table VI. 


TABLE VI 
SCALING CONSTANTS 


Scaling Constants FCS Radar Gas Turbine 





Ihe utility functions, using the scaling constants 


from Table VI, are: 


(a) for the Mark 92/2 FCS problen; 


E ТЕТЕ ТЕН ЕСІГІ 1€0-02u. GE.) 7 


(b) for the AN/SPS-49 radar problem; 
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NEL OLEO UG JE 9-199, (5) JL 17-90-36. (x JJ 
and 


(c) for the LM 2500 gas turbine engine problen; 


1+1.37u (X) =[ 1+0.1 140.35 1+0.6 
+ sn eg: Ора осока 


The utility of са: nz represents the utility of the 
attribute future usefulness, Pae represents the utility 
of the attrikute schedule and E represents the utility 


of the attribute cost. 


The second step in the decision analysis process has 
been completed. Utility functions іп the multiplicative 
form have been derived so that utility numbers can be 
expressed for each of the consequences indicating the 
decision-maker's preferences for these consequences. To 
conplete formulation of the LBTS problem, judgmental 
probabilities are assessed for the uncertain events of time 
to accomplish tests, Certification, Operational Assist and 
DSARC III. | 


C. ASSESSMENT OF JUDGNENTAL PROBABILITIES 


By informal correspondence, the PF Assistant Test and 
Evaluation Manager was asked to assess the probabilities for 
the uncertain events for the installation and testing cf the 
three suksystems under analysis. Table VII summarizes the 
results of his reply. The probabilities indicated in this 
table are assigned to the appropriate chance forks cf the 


three Сесіспоп trees. 
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TABLE VII 
JUDGMENTAL PROBABILITIES FOR LBTS PROBLEM 


if actual equipment installed : 


Probability of: | MK 92 ECS SPS-49 LM 2500 ENG 
sufficient time for 

Certification 0.4 0.8 0.01 
passing Op Assist, . | 

given passed Certification 0.7 0:7 0.8 


passing 2: Assist. 
1ven no Certification 
one 0.2 0.6 0.3 


passing DSARC III if 
passed Certification 
and Op Assist 0.8 0.7 0.8 


passing DSARC,III, if 
passed pa clc. but 
no Certification done 0.1 0.5 0.2 


If part of system simulated: 


mE nn RA eee ge oe ee ee ee ee ee ee ee ee 


Probakility of: MK 92 FCS 
sufficient time for 
Certification 0.4 


passing Certification 
ıf tine available 0.5 


passing Op Assist, . 
given passed Certification 0.4 


passing Sip Assist. 
іуеп no Certification 
one 0.2 


passing DSARC III if 
passed Certification 
and Op Assist 0.6 


passingfDSARC PII if 


passed Op Assist but 
no Certification done 0.1 
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C. If all of system simulated: 


Cu RUNG UNE CGU «иши cub Un «туні» «ШЕН comb ee cup ecu CURED uem cub cmm —-— 


Probability of: MK 92 FCS SPS-49 LM 2500 ENG 
1. sufficient time for 

Certification 0.2 0.7 0.4 
2. passing Certification 

if tinme available 0.3 0.7 0.35 
3. passing Of Assist, . | 

given passed Certification 0.2 0.7 0.4 


4. passing Op Assist |. 

lven no Certification 

one 0.1 0.6 0.2 
5. passing DSARC III if 

passed Certification 

and Op Assist 0.1 0.5 25 
6. passing DSARC III if 


passed Op Assist but 
no Certification done ОЕ 05 0.4 0.5 


Probability of: MK 92 FCS S PSI 
1. passing DSARC III 0:2 0.6 


D. SUMMARY OF CHAPTER 
/ 


The systematic process of decision analysis was used to 
formulate the problem of deciding whether to install actual 
subsystems, partially simulate them or simulate the entire 
subsystem at the Patrol Frigate LBTS. The alternative of 
not installing the subsystem at the LBES was also 
considered. Three subsystems were chosen from the Risk 
Watch List for the analysis. They are the Mark 92 mod 2 
fire control system, the AN/SPS-49 radar and the LM 2500 gas 
turbine engine. MIS COMO AO this a chapter, the 
structure of the problem was formulated in terms of a 
decision tree. Figure 6 illustrates a typical branch of the 
decision tree for the installation of actual equipment, 


partially simulated equipment or simulated equipment. 
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Figure 7 illustrates the "no installation" alternative 
branch of the decision tree. Section B. contained a 
discussion of the second step in the decision analysis 
process, assessment of the decision-maker's preferences for 
the consequences. They are expressed in terms of utility 
numbers. A utility function which assigns the utility 
numbers to the consequences was formulated in the 
multiplicative form. The equations derived are shown in 
this section. Section C. comprised the third step in the 
decision analysis process, assessment of the judgsental 
probabilities for each chance fork of the decision tree. 
Table VII lists these probabilities. 


Chapter V contains a discussion of the final step in the 
decision analysis process, determination of the best courses 
of acticn using the information formulated in this chapter. 
The word "final" implies that the process is completed when 
the best course of action is determined. AS was pcinted 
out, the decision analysis process 15 iterative. This 
thesis represents the initial analysis of a complicated 
problem. In the conclusion, Chapter VI, some suggestions 
for further research to refine the analysis of this thesis 


are made. 
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V. PF LBTS PROBLEM SOLUTION RESULTS 


ane ee «UHREN GE «лынып A: ee eee ee ee ee o: AGED GENE A CO E E ee UNES O шз шр сш URGED > NES аны» 


All of the information required to solve the PF LBTS 
probiem, including a discussion of the alternatives andá 
conseguences for each subsystem, and cost, scheäule and 
usefulness considerations have been provided in the 
preceeding chapter. The functions which transform the three 
attributes into a scalar measure of utility for the outcomes 
were derived, and judgmental probabilities were assessed for 
each uncertain event of the decision tree. All that remains 
is to structure the decision tree, asSign the appropriate 
utility to each end point, insert the judgmental 
prokabilities at the chance forks and determine the expected 


utility values for each alternative of the tree. 


in this chapter, the solution results for the AN/SPS-49 
radar, the Mark 92/2 FCS and the LM 2500 gas turbine 
problems will be discussed separately. In each case, the 
decision tree is provided before the results are discussed. 
The decision trees presented are analyzed by computing the 
expected utility of each chance fork and the selecticn of 
the greatest utility at each decision fork. The process is 
repeated for each level of the tree until the starting 
decision fork is reached. The alternative with the greatest 
expected utility is selected as the best course of action. 
A computer program written in the BASIC language, was used 


to perform the calculations. 
А. SPS-49 RADAR 


The decision tree for the AN/SPS-49 radar problen 165 
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exhibited in Figure 8. The alternatives which were 
considered are: installation of the actual equipment, 
installation of simulated eguipment or no installation. The 
assessed judgmental probabilities are indicated near each 
"Pass" and "Fail" of the chance forks. Table VIII lists the 
outcomes for each of the twenty-six consequences and 
provides the utility for each conseguence. The utility 
functions derived for the AN/SPS-49 radar problem are shown 


in section B.3. of the previous chapter. 


The sclution results are: 
(1) the expected utility of the alternative, 
installation of the actual equipment, is 0.76, 


(2) the expected utility of the alternative, simulation 


of the equipment, is 0.72 and 


(3) the expected utility of the alternative, no 
installation, is 0.81. 
These results indicate that the recommended course of action 
is not to install the SPS-49 radar at the Combat System 
LBTS. 
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DECISION TREE SPS-49 RADAR 
ACTUAL EQUIPMENT AND NO EQUIPMENT BRANCHES 


Consequence 
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1 
DSARC 
Pass 
II em 2 
o и Сапсе 
Ға11 
0.8 m Do over 
Pass 
0.7 3 
Op Assist Cancel 
Q P - Ра11 
Pass г oO over 
к 5 
© Certification 
SAmulate Cancel 6 
Fail 
0.2 m )O OVer г 
Time 
" ctual 0.8 
equip. Е 
o time 
032 9 


Cancel 
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FIGURE 8 
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DECISION TREE SPS-49 RADAR 
SIMULATE BRANCH 


Consequence 
numter 
15 


Р 
III yë 16 
Q ` Сапсе1 


Ға11 
9355 mm Do over 






0.7 17 
Op Assist ( ) Cancel 18 
Fail 
Pass 03 ж Do over 
0.7 
19 
Certification 
© | Сапсе1 20 
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0.3 E Do over 
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Time 
(E—— ——CY 0.7 DSARC 22 
Simulate Е ІТІ 
О time 
7» 3 23 
Cancel | 
Pass 
0. 24 
Op. ; 25 
Assist Fail Cancel 
' 0.4 
ш Do over 
26 


FIGURE 8 (continued) 
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TABLE VIII 


OUTCOMES AND RESULTS OF SPS-49 DECISION TREE 


CONSEQUENCE OUTCOMES , UTILITY 
Number Usefulness Schedule Cost 
(scalar) (nonth) (nillion) | 
1 1.5 25 0.93 
2 0 4 9e 0.55 
3 3 4 Зе 0.68 
y 0 4 ai) 50 55 
5 3 ü i 0.68 
6 0 4 3:53 0.55 
7 3 4 ва 0.68 
8 3 4 2.5 0.72 
9 0 ü За» 0.55 
10 3 4 3.93 0.68 
11 0 4 929 0255 
12 3 4 345 0.68 
13 0 0 0 0.90 
14 3 Ц 3:5 0.68 
15 1 1.3 1 Ш> 92 
16 0 4 2 0.60 
17 1 4 2 0.67 
18 0 4 2 0.60 
19 1 4 2 0.67 
20 0 4 2 0.60 
21 1 4 2 0.67 
22 1 4 1 0.70 
23 0 4 2 0.60 
24 1 4 2 0.67 
25 0 4 2 0.60 
26 1 4 2 0.67 





SOLUTION RESULTS 


ALTERNATIVE T'OEXPEGTED US TI EDY 
IS Actual, 0.76 
2 Sinulaited DET 
3. None 0.81 


712 





В. MARK 92 MOD 2 FCS 


The decision tree for the Mark 92/2 FCS problem is 
exhibited in Figure 9. The alternatives which were 
considered are: installation of the actual equipnent, 
partial simulation of the equipment, complete simulation of 
the equipment and no installation. The assessed 
probabilities are indicated near each "Pass" and "Fail" of 
the chance forks of the tree. Table IX lists the outcomes 
for each of the consequences and provides a range of 
utilities for each consequence. Recall that a family of 
utility curves was established for the schedule attribute. 
Hence, fcr the fire control system problem, four different 
sets of points for the schedule attribute were used to check 
the sensitivity of the shape of these curves on the 
solution. The utility functions derived for the fire 
control system installation problem are shown in section 


B.3. of the previous chapter. 

The sclution results are: 

(1) the ex pected utilities for the alternative, 
installation of the actual equipment, range from 0.38 to 


0.63, 


(2) the expected utilities for the alternative, partial 


simulaticn cf the equipment, range from 0.50 to 0.78, 


(3) the expected utilities for the alternative, 


Simulaticn cf the equipment, range from 0.40 to 0.70 and 
(4) the expected utilities for the alternative, no 


installation, range fromi0.43 to 0.67. 


Dhesesresuigesımascate that tho recommended course of action 
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is to partially simulate the fire control systen to be 
installed at the Combat Systen LBTS. They are consistent 
over the entire range of the utilities. The different 
shapes of the schedule attribute utility curve do not affect 


the solution results. 


С. LM 2500 GAS TURBINE ENGINE 


The decision tree for the gas turbine engine problen is 
shown in Figure 10. Two alternatives were considered. They 
are: installation of the engines or simulation cf tne 
engines. The assessed judgmental probabilities are 
indicated near each "Pass" and "Fail" of the chance forks of 
the tree. The outcomes for each of the twenty-four 
Consequences and a range of utilities for the outcomes are 
provided in Table X. As with the fire control system 
problem, a family of curves was established to represent the 
utility curve of the schedule attribute. The range of 
utilities represent four different sets of points used to 
check the sensitivity of the shape of these curves on the 


solution results. 
The sclution results are: 


(1) the expected utilities of installing the gas turbine 


engines range from 0.19 to 0.35 and 


(2) the expected utilities of simulating the gas turbine 
engines range from 0.4 to 0.64. 
The recommended course of action is to simulate the gas 
turbine engine at the Propulsion System LBTS. These results 
are consistent over the entire range of the utilities. The 
different shapes of the schedule attribute utility curve do 


not tle €rHoc-solutvon results. 
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DECISION TREE MARK 92/2 ЕС5 
ACTUAL EQUIPMENT AND NO EQUIPMENT ERANCHES 
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FIGURE 9 
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DECISION TREE MARK 92/2 FCS 
PARTIALLY SIMULATE BRANCH 
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FIGURE 9 (continued) 
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DECISION TREE MARK 92/2 FCS 
SIMULATE BRANCH 
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FIGURE 9 (continued) 
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TABLE IX 


OUTCOMES AND RESULTS 


FCS DECISION TREE 
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UTILITY 
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DECISION TREE LM 2500 GAS TURBINE ENGINE 
ACTUAL EQUIPMENT BRANCH 
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LECISION TREE LM 2500 GAS TURBINE ENGINE 
SIMULATE BRANCH 
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TABLE X 


CONSEQUENCE NUMBER, 
GAS TURBINE DECISION TREE 


NR OUTCOHES 
Use Sched Cost 
1 3 10 14 
2 0 16 15 
3 3 16 15 
4 0 16 15 
5 3 16 15 
6 0 16 15 
7 3 16 15 
8 3 16 14 
9 0 16 15 
10 3 16 15 
11 0 16 15 
12 3 16 15 
13 1 10 3 
14 0 16 4 
15 1 16 4 
16 0 16 4 
17 1 16 4 
18 0 16 4 
19 1 16 4 
20 1 16 3 
21 0 16 4 
22 1 16 4 
23 0 16 4 
24 1 16 4 
ALTERNATIVE 


1 
0.51 
0.11 
0.19 
0.11 
0.19 
0.11 
0.19 
0.22 
0.11 
0.19 
0.11 
0.19 
0.82 
0.38 
0.43 
0.38 
0.43 
0.38 
0.43 
0.46 
0.38 
0.43 
0.38 
0.43 


UTILITY 

2 3 

0.51 0.51 
0.14 0.20 
0.22 0.29 
0.14 0.20 
0.22 0.29 
0.14 0.20 
0.22 0.29 
0025 0.31 
0.14 0.20 
0.22 0.29 
0.14 0.20 
0.22 0.29 
0.82 0.82 
0.42 0.50 
0.47 0.55 
0.42 0.50 
0.47 0.55 
0.42 0.50 
0.47 0.55 
0.50 0.58 
0.42 0.50 
0.47 0.55 
0.42 0.50 
0.47 0855 


SOLUTION RESULTS 


EXPECTED UTILITY 


OUTCOMES AND RESULTS 


0.51 
0.25 
0.35 
0.25 
0.35 
0.25 
0.35 
0.38 
0.25 
0.35 
0.25 
0.35 
0.82 
0.57 
0.63 
„57 

0.63 
USO 
0.63 
0.66 
0.57 
0.63 
0.57 
0.63 





1. Actual 
D. Simulated 


1 
0. 19 
0.45 
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2 
0.23 
0.49 


3 
0:29 
0.56 


О 
0.64 








ҮІ. SUMMARY AND RECOMMENDATIONS 


eee a -—— im pm EEE Zu Apo тб O E E A A A AR эши Cu 


Іп this thesis, decision analysis methodology was 
applied to the problem of some project management trade-off 
decisions required for the Patrol Frigate Land Based Test 
Sites. In particular, the question of what PF subsystems 
should be installed and tested at the LBTS's was studied. 


This chapter summarizes the content of the thesis, 
presents recommendations and provides suggestions for 


further research. 


А. THESIS SUMMARY 


Chapter II provided the background of the Patrol Frigate 
Land Based Test Sites. The DSARC review process was 
introduced along with the required test and evaluation 
reguirements necessary in each stage of the Patrol Frigate's 
acquisition cycle. The problen of what the composition of 
subsystens should ke in the Combat and Propulsion System 
LBTS so that adequate test and evaluation with satisfactory 
results can be supplied to DSARC III, was provided as an 
example to which decision analysis methodology сап be 


applied. 

Chapter III supplied the decision analysis methodology 
required to formulate and solve the РЕ LBTS problem 
described in Chapter II. The steps in the formal process of 


decision analysis were discussed. They are: 


(1) structuring the problem in terms of a decision tree, 
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(2) evaluation of the consequences in utility numbers, 


(3 assignment of probabilities to the branches of the 


chance forks of the decision tree and 


(4) determination of the optimal strategy by the 
"averaging out and folding back" process. 
It was pointed out that the formal analysis is an iterative 
process. First, a broad description of the problem with 
rough assessments of the utilities and probabilities is made 
to determine whether any branches at decision forks can be 
eliminated or added. Next, the measurements are refined and 
certain forks added or eliminated. This cycle is repeated 
until there is satisfaction with the results of the 


analysis. 


The decision tree represents the chronolcgical 
arrangement of the choices or alternatives ccntrolled ty the 
decision-maker and the choices determined by chance. The 
evaluaticn procedures for the assignment of utility numbers 
to each conseguence were discussed. A conseguence is 
represented by a path through the complete decision tree. 
The utility numbers are generated by a utility function 
which transforms or maps the attribute descriking the 
outcome of a consequence into a scalar value. The 
conseguences for a complex problem are generally described 
by multiple attributes which complicate the construction of 
the utility function. Simplification of the multiattribute 
utility function was discussed utilizing the properties of 
utility independence, pairwise preferential independence and 
pairwise marginality. Table I summarized the properties 
necessary for each type of utility function simplification. 
The assessment of the judgmental probabilities tc each 
chance fcrk of the decision tree was. then outlined. With 


all the necessary information available for the decision 
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tree analysis, the technique of "averaging out and folding 
back" to determine the expected utility of each alternative 
was described. The alternative with the greatest expected 


utility is chosen as the recommended course of action. 


The Patrol Frigate LBTS problem наз formulated in 
Chapter IV. This formulation encompassed the initial 
iteration of a formal decision analysis process. The Mark 
92/2 fire control system, the AN/SPS-49 radar and the LM 
2500 gas turbine engine were the PF subsystems examined. In 
the testing and evaluation process of these subsystems at 
the LBTS, the Project Manager worried about the installation 
and testing costs, the schedule for adequate test and 
evaluation, the uncertainty of passing DSARC III review and 
the usefulness of the subsystem at the LBTS to the program 


after DSARC III. His alternatives, in general, were to: 
(1) install actual or simulated equipment at the LETS, 


(2) conduct at-sea testing of each subsystem during 


Operational Evaluations, 
(3) monitor tests at the vendor's activity and at sea or 


(4) conduct no further testing. 
In passing DSARC III, the Project Manager is concerned with 
the uncertainty of passing Certification and Operational 
Assist .prior to the DSARC III review. He is also uncertain 
as to the availability of time to accomplish these test 
requirements. These uncertainties were incorporated into 
the decision tree structure as chance forks in the 
appropriate order along the paths of the tree. The 
alternatives chosen for the fire control system were to 
install the actual equipment, partially simulate the fire 
control system, simulate all of the equipment or not to 


inec luder the fire control system at thè Combat System  LBTS. 
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The alternatives chosen for the AN/SPS-49 radar were to 
install the actual equipment, simulate the radar system or 
not to include the radar at the Combat System LBTS. Two 
alternatives were chosen for the LM 2500 gas turbine engine. 
They were to install the engine or simulate the engine at 
the Propulsion System LBTS. 


Cost, schedule and future usefulness were selected as 
the attributes describing the outcomes of each consequence 
of the tree. These attributes for the three subsystem 
problems were determined to have the properties of first 
order utility independence and pairwise preferential 
independence. As a result, the utility functions were 
expressed ig the multiplicative form. The  judgmental 
probabilities assessed for each chance fork are listed in 
Table VII. 


With the decision tree structure defined, the utility 
functions derived and the chance fork probabilities 
assessed, each subsystem problem was structured in terms of 
a decision tree in Chapter V. A computer program, written 
in the BASIC language, was constructed to evaluate the 
conseguences of the tree in utility numbers and to "average 
out and fold back" the tree to arrive at the expected 
utility of each alternative considered. Tables VIII, IX and 
X list the results. For the SPS-49 radar, the recommended 
alternative was not to install the radar at the Combat 
System LBTS. For the LM 2500 gas turbine engine, the 
recommended alternative was to simulate the engine at the 
Propulsion System  LBTS. Hor thei Mark 92/2 FCS, the 
recommended alternative was to partially simulate the fire 


control system. 
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В. RECOMMENDATIONS 


Тһе decision tree analysis yielded several 
recommendations which contrast with the alternatives 
selected by the PF Project Manager. In actuality, it has 
been determined that the SPS-49 radar and the LM 2500 gas 
turbine are to be installed at their respective Land Based 
Test Sites [Ref. 5]. A result of this thesis is that the 
radar should not be installed and the gas turbine engine 
should be simulated. It is recommended that these 
alternatives should be further considered by the PF Project 
Manager. In the case of the SPS-49 radar, it is nct too 
late to consider the alternative of not installing the 
subsyStem at the  LBTS. Installation of the LM 2500 gas 
turbine engines is already in progress and it is, perhaps, 
too late to consider its simulation. The alternative 
selected for the Mark 92/2 FCS was to partially simulate the 
fire control system which is in agreement with what was 
actually decided. This alternative, however, was primarily 
selected by the Project Manager because of the location of 
the Combat System LBTS where, out of necessity, the missile 


launchers and gun systens have to be simulated [Ref. 5]. 


The alternative of not installing the LM 2500 gas 
turbine at the LETS was not considered in the analysis. If 
there is no engine, then there is no propulsion systen. 
This alternative, if considered, really leads tc tne 
question of the desirability of a propulsion system LBTS. 
Rough data was available for consideration of the 
alternative of installing no engine and the expected utility 
of this alternative was computed. It was found that the 
expected utility was greater than that of the other two 
alternatives considered. This result would lead directly to 


the recommendation that a Propulsion System LBTS would not 
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be required. It is recommended that for future ship 
acquisition projects, the desirability of a propulsion 
system LETS be closely studied. A propulsion system LBTS 
Should not be preordained by the spirit of existing DOD 


Directives. 


The analysis of this thesis has shown some results which 
are in contrast to the alternatives actually selected. This 
thesis has shown that decision analysis can be a viable 
decision-making tool for the project manager. It is a 
stimulus to think about new alternatives. It allows the 
decision-maker to scrutinize a problem as an organic «hole. 
Decision analysis provides the decision-naker with the means 
to distinguish his preferences for consequences frcm his 
judgments abcut uncertainties [Ref. 11]. Decision analysis 
also allows the project manager to demonstrate that his 
decisions were not made on a casual basis and that a 
formalized decision approach was used. He can use the 
analysis to communicate the rationale of the chosen strategy 
to obtain support for his decision. It is recommended that 
decision analysis methodology be incorporated more fully 
into the project management environment. Raiffa [Ref. 11] 


states: 


{ IT my personal, opinion one part of the 
justification for adoptina the methodology of decision 
analysis is that the underlying behavioral assumptions 
ағе срез етігі авсссопоираста os енй JustificatuicneJs 
that is metnodology is an operational mode of analysis 
(at least for many рама ситат бе classi as e Si 
andme them Гъпаф драго ог thel justification isf what wou d 
you do otherwise?! 


C. SUGGESTIONS FOR FURTHER RESEARCH 


This thesis represents an initial analysis СЕ а 


complicated problem. Further research is necessary to 
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refine the probability and utility measurements. In 
particular, the effect of performance as an attribute should 
be further studied. This is a very complicated area and 
would make a thesis in its own right. The probability 
assessments were made at the project manager level. These 
probabilities could be further refined by assessing the 
judgments of technical personnel located at the LBTS and at 
the level of the Office of the Chief of Naval Operations. 
These assessments could then be compared and refined fer the 


next iteration of the analysis. 


The tools developed in this thesis can be applied to 
other areas of system acguisition program management. Some 


suggested areas are: 


(1) an analysis of procurement contract alternatives of 


a major weapons system acquisition, 


(2) an analysis of the different options in the 
selection of subsystems necessary for a ship or weapons 


System, 


(3) an analysis of the alternatives among different 
types of hull platforms for a designated weapons system such 
as the AEGIS system and 


(4) an analysis of a centralized procurement plan for 
Material other than government furnished material versus the 
alternative of independent procurement of materials Ву 


“Several contractors for the same weapons system acquisition 


project. 


D. A CONCLUDING REMARK 


It is hoped that this thesis has generated some interest 
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in decision analysis in the project management environnent. 
The decisions reguired are complex and involve a great deal 
of uncertainty in an area where cost and schedule  over-runs 
make it desirable that the option selected is the best 
possible. The Congress and concerned citizens demand it. 
Decision analysis is a tool that managers in the procurement 
of major weapons systems can use. The ideas, of decision 
analysis are relatively new but they are too promising to 
ignore. Decision analysis can be  experimented with by 
applying it to selected problems on paper and comparing the 
results with decisions that are actually made. If the 
results аге favorable or raise some valid questions not 
considered, as in the example provided by this thesis, then 


these techniques should be adopted for use in the future. 
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APPENDIX A 


The steps in evaluating the paraneters of the utility 
function for the Mark 92/2 FCS decision tree are outlined in 
this appendix. This involves evaluation of the scaling 
constants K, ща a and КА of the utility function 


3 
1 + Ku(X) = elt [i+ Б MESE 


Section B.3., Chapter IV, contains a description of this 


process. 
А. PREFERENCE RANKING OF ATTRIBUTES 


The preferences for the three attributes were ranked by 
the PF Test and Evaluation Manager as: cost is preferred to 


schedule, and schedule is preferred to usefulness [ Ref. 19]. 
В. DETERMINATION OF LINEARLY INDEPENDENT EQUATIONS 


To establish four linearly independent : eguations, 


equation (3), Chapter IV, 


3 
1+ K = 1 + Kk 
gd de 
and equation (4) 
3 
prom Ke ‚al (1 * Kk.) / (1% КК), 
J = 1 2 
are used. The utility value m , where j = 1,.,3, is first 
j^ 
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determined by the assessment of gambles of the alternatives 


listed below. 


ALTERNATIVES 





dd ы С о > 











USEFULNESS SCHEDULE COST 
(scalar) (month) (million) 

2:5 

29 

Ns» 

18.9 

TONS 


The gambles are assessed per the following table: 


GAMBLE 





D is 


B is 


C is 


D is 


B is 


C is 


indifferent 


indifferent 


indifferent 


indifferent 


indifferent 


indifferent 


indifferent 


indifferent 


indifferent 


to 


to 


to 


to 


to 


to 


to 


to 


to 


The utility ог В, on 


‚ 15 selected 


selected as 0.44. 


<А,.7,Е> mn, = 0.70 

с ТЪВЕФ Да = 0.44 

<А,.6,Е> Ip = 0.88 

<B ps ре m = 0.66 
2M 

<В. оше? m = 0.44 
3 

<А оа ро. | T - 0.88 

SD. O RE? ща = 0.70 

<A, ШЕ Е> mo = 0.40 
3 

GN, .9, ED Др - 0.90 


is selected as 0.88. The utility of 


as 0.66. Thefzutlity оғ р; Ik 1S 


5/7, 





С. SOLUTION OF SCALING CONSTANTS 


With ^ Ще апа дЕ assessed, the scaling constant K can 


be determined from equation (6), Chapter IV, 


i 


3 
(1 + K)* | (1 + m K) 
351 j 


(1+.88K) (1%.66К) (1+.44K). 


Hence, 
K (.26K?+.26K-.02) = 0, 


where К has the values 0, -1.08 and 0.07. Keeney [Ref. 13] 
shows that the value K which is nonzero and greater than -1 


is chosen, therefore, K = 0.07 is selected. 


Since the constant K is known, M Ки and a can be 


determined from equation (7), Chapter IV, 


Кк = (1-а) / (ET merk): 
2 2 2 
Therefore, 

К, = (1 - .88) / [1 + .88(.07) ] 
— 022790 

A A A 
80:42 

ала | 
к. = (1 - .44) / [1 + .44(.07)] 


0.64. 
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APPENDIX B 


SCALING_CONSTANT EVALUATION SPS-49 RADAR 


mi srl cium ee E E e T ee ee ee ee ee TR CTO ECO — NES Оңы ЫК шы „ OS ld: ee EDS 


The steps in evaluating the parameters of the utility 
function for the SPS-49 radar decision tree are outlined іп 
this appendix. This involves evaluation of the scaling 


constants K, ST = and ES of the utility function 


3 
1 + Ku(X) = 11 [1 + SEU CUN) ЩЕ 


Section B.3., Chapter IV, contains a description of this 


process. 
А. PREFERENCE RANKING OF ATTRIBUTES 


The preferences for the three attributes were ranked by 
the PF Test and Evaluation Manager as: cost is preferred to 


schedule, and schedule is preferred to usefulness [ кеё. 19]. 
B. DETERMINATION OF LINEARLY INDEPENDENT EQUATIONS 


To establish four linearly independent equations, 
equation (3 Сһартес Ty, 


3 
1 + К < 1 + КК 
"T 


and equation (4) 


3 
1 + Пк 1 + КК 1+ кк), 
m E Р. 


are used. The utility value m , where j = 1,.,3, 1s first 
J 
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determined by the assessment of gambles of the alternatives 


listed below. 










ALTERNATIVES USEFULNESS SCHEDULE COST 
(scalar) (month) (million) 

А 125 

E (95 

е 125 

D 3,5 

E 3.5 


Тһе gambles are assessed per the following table: 


GAMBLE UTILITY (m ) 
3 


B is indifferent to <A,.9,E> 


С 15 indifferent to <A,.8,E> nm = 0.80 
D is indifferent to <А,.5,Е> | па = 0.50 
B is indiffernt to <A,.5,c> S. - 0.9 

С 15 indifferent to <B,.8,E> P: - 0.72 
D is аа Ты ТТЫ to <В, 55 ДЕ> ще = 0.45 
B is indifferent to <A,.8,D> ay = 0.90 
Greasy indifferent to <B,.7,D> ща = 0.78 
D is indifferent to "NC De = 0.48 


. The utility Of B, T selected as 0.9. The utility of C, 
m „ is selected as 0.8 and the utility of D, x is selected 


ais# 025: 
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С. SOLUTION OF SCALING CONSTANTS 


With np ДИ апа De assessed, the scaling constant K can 


be determined from equation (6), Chapter IV, 


(1% К)2 


3 
(1 + m K) 
151 3 
(1%.9К) (1%.8К) (1+.5K). 


Непсе, 
K (.36K2+.57K+.2) = 0, 


where K has the values 0, -1.06 and -0.52. Keeney [Ref. 13] 
shows that the value K which is nonzero and greater than -1 
is chosen, therefóre, K = -0.52 is selected. 


Since the constant K is known, E > and КЕ сап Бе 


determined from equation (7), Chapter IV, 


k = (1-m) (1 * n E). 
J J J 
Therefore, 

“ AAA A A (2.92) | 
= 0.19, 

a ЕЕ ОИЕ 169.52) ] 
= 0.34 

and 

22 ШЕГУ СГ. 5.52) | 

= 0.68. 





APPENDIX С 


SCALING CONSTANT EVALUATION GAS TURBINE ENGINE 


= oc; слышал ыс” с... ES E 10 RR re ee a ee a a ELLE lI qc il 


The steps in evaluating the parameters of the utility 
function for the gas turbine engine decision tree are 
outlined in this appendix. This involves evaluation of the 
scaling constants K, у = and z of the utility function 


3 
1 + Ku(X) = ¡E + AE 


Section B.3., Chapter IV, contains a descriptior of this 


process. 
А. PREFERENCE RANKING OF ATTRIBUTES 


The preferences for the three attributes were ranked by 
the PF Test and Evaluation Manager as: cost is preferred to 


schedule, and schedule is preferred to usefulness (Ref. 19]. 
В. DETERMINATION OF LINEARLY INDEPENDENT EQUATIONS 


To establish four linearly independent equations, 


equation (3), Chapter IV, 


3 
1 + К + 1#+ КК 
т а 
and equation (4) 
3 
1 +m K = Jf (1 + Kk.) / (1 + КК), 
J 1= 1 J 
агер uscd. The utility value ш , where j = 1,.,3, is first 
J 
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determined by the assessment of gambles of the alternatives 


listed 


ALTERNATIVES 


below. 


т о с) ш m 










USEFULNESS SCHEDULE cosT 
(scalar) (month) (million) 
3 
15 
15 


The gambles are assessed per the following table: 


B is 


Deis 


С 15 


В 15 


D is 


eis 


The utility of B, NE 


С m 
ша 


GAMBLE 





indifferent 
indifferent 
indifferent 
indifferent 
indifferent 
indifferent 
indifferent 
indifferent 


indifferent 


Cr 





to 


to 


to 


to 


to 


to 


to 


to 


to 


, is selected 


selected as 0.35. 










<А,.5,Е> ШЕ = 0.5 
UTC n. - 0.85 
uU) EX e = 0.3 
Ee OR D> ща - 0.58 
ЧАБ . 7 ла> np = 0.82 
SONO „Е2 un = 0.34 
<В,.6,Е> D» - 0.51 
<Д,.8, Б> 1 = 0.8 
<в,.4,Е> ШЕ - 0.34 


is selected as 0.85. The utility of 


as 0.55 and the utility of D, s 15 
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С. SOLUTION OF SCALING CONSTANTS 


Hith п, ШЕ and > assessed, the scaling constant K can 


be determined from equation (6), Chapter IV, 


(1 + K)2 


И 


3 

| (1 + m_K) 

B. 
(1+.85K) (1%.55К) (1%.35К). 


Непсе, 
K (.16K2-.04K-.25) = 0, 


where K has the values 0, -1.11 and 1.37. Keeney [Ref. 13] 
shows that the value K which is nonzero and greater than -1 
is chosen, therefore, K = 1.37 is selected. 


Since the constant K is known, EN E and m can be 


determined from equation (7), Chapter IV, 


К Уор) 7 (1 з Қ). 
J J 2 
Therefore, 

Ка = (1-.85) / | 1+.85 (1.37) ) 
= 0.07, 

> a Oey at woo (be ST) | 
= 0.26 

and 
= = (1-.85) И | 1+.35 (1.37) | 


0.44 
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